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Method and Apparatus for Automated Detection of Target 
Structu res from Medical Images Usincr A 3D Morphological 

Matching Algorithm 



Cross Refer ence and Priority Claim to Related Application: 
This application claims the benefit under 35 U.S.C. § 
119(e) of provisional patent application Ser. No. 60/43 0,954 
entitled "Method and Apparatus for Automated Detection of 
5 Pulmonary Nodules from Computed Tomography (CT) Images Using A 
3D Morphological Matching Algorithm", filed December 
2002, the disclosure of which is incorporated by reference 
herein. 

10 Field of the Invention: 

The present invention relates to a system for computer- 
aided diagnosis (CAD) of target structures using digital 
medical images. In particular, the present invention ^relates 
to the application of CAD techniques to CT images to thereby 

15 automate the detection of target structures such as pulmonary 
nodules. 
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Background of the Invention: 

Lung cancer is the second most commonly diagnosed cancer 
in the United States, and other cancers frequently metastasize 
to the lung parenchyma as pulmonary nodules. Chest CT is the 
most sensitive diagnostic imaging modality for detecting lung 
mass or lung nodules. With widely accepted spiral CT and 
newly developed multislice CT techniques, the sensitivity in 
detecting pulmonary nodules has further improved. 

Recently, spiral CT techniques have been applied to 
screening for lung cancer in high risk populations, and it has 
been shown to be useful in the early detection of lung 
cancers. See Henschke et al., Early Lung Cancer Action 
Project: overall design and findings from baseline screening, 
Lancet; vol. 213, pages 723-26 (1999), the disclosure of which 
is incorporated herein by reference. High resolution, three- 
dimensional (3D) imaging of a patient's thorax allows for the 
evaluation of small nodules at early stages. With follow-up 
scans, any changes in nodule number and nodule size can be 
20 assessed. 

High resolution CT scans of the thorax generate a large 
amount of data - more than 250 images with a slice thickness 
of 1 mm. If CT scanning becomes widely practiced for lung 
cancer screening, an enormous demand will exist for physicians 
to review and interpret the scan images. To help alleviate 
such a burden in a high volume screening scenario, a computer 
system for automated nodule detection would be useful. 
Moreover, CAD systems can help improve the quality of a 
radiologist's performance in detecting pulmonary nodules. 

Various CAD systems for pulmonary nodule detection using 
CT images have been proposed since the introduction of the 
first system by one of the inventors herein more than a decade 
ago (see Bae et al . , Computer-aided detection of pulmonary- 
nodules in CT images, Radiology 1991; 181 (P) : 144, the entire 
35 disclosure of which is incorporated herein by reference) . The 
majority of these CAD systems use detection algorithms based 
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on two-dimensional (2D) morphologic features in each slice or 
serial 2D methods that examine connectivity of features in 
adjacent slices (see Reeves and Kostis, Computer-aided 
diagnosis for lung cancer, Radiol Clin North Am 2000; 38:497- 
5 509; and Armato et al ., Automated detection of lung nodules in 
CT scans: preliminary results, Med Phys 2001; 28: 1552-61, 
the disclosures of both of which are incorporated herein by 
reference) . Recently, two CAD methods have been reported that 
focus on 3D algorithms which take advantage of high resolution 

10 volumetric CT data obtained with multislice CT systems (see 

Qian et al . , Knowledge -based automatic detection of multi-type 
lung nodules from multi -detector CT studies, SPIE 2002; 4684: 
689-697; and Fein et al . , Automatic detection of lung nodules 
from multi-slice low-dose CT images, SPIE 2001; 4332: 1828- 

15 1835, the disclosures of both of which are incorporated herein 
by reference) . 

Summary of the Invention: 

From this array of developing CAD techniques, a need in 
20 the art exists for an efficient algorithm that combines both 
3D and 2D morphologic operations in the segmentation, 
detection, and classification of target structures. Further, 
different types of target structures, such as different types 
of pulmonary nodules, require detection algorithms specially 
25 tailored therefor to promote successful diagnosis. As such, a 
need in the art exists for a CAD technique that has the 
capability of identifying which image regions should undergo 
different detection processing and further has the capability 
of performing the different detection processing on the 
identified regions. For example, in the detection of 
pulmonary nodules, different types of pulmonary nodules have 
unique characteristics that make separate processing thereof 
important for successful diagnosis. The three types of 
pulmonary nodules of interest are isolated nodules, 
perivascular nodules, and pleural nodules. 
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Isolated nodules are nodules that are dissimilar to 
adjacent blood vessels. Isolated nodules exhibit relatively 
high contrast with the surrounding lung, and as such, can be 
easily recognized in a 2D slice without comparing adjacent 
5 slices. 

However, perivascular nodules are not easily 
differentiated from the nearby blood vessels because they are 
closely associated with adjacent blood vessels and because 
some blood vessels that are perpendicular to the imaging plane 
10 exhibit a similar appearance as perivascular nodules in 2D 

slices. As such, comparison between adjacent slices (i.e., 3D 
operation) may be essential to the successful detection of 
perivascular lung nodules. It should be noted that the term 
"perivascular nodules' 7 as used herein encompasses 
15 juxtavascular nodules. 

Pleural nodules, which are pulmonary nodules attached to 
the pleural surface, may be easily detected in a 2D slice by 
the human eye. However, an additional operation is generally 
required to properly delineate them from the pleural boundary. 
20 in an effort to develop an efficient technique capable of 

accurately detecting target structures such as pulmonary 
nodules, the inventors herein have developed a method for 
automated detection of target structures shown in digital 
medical images, the method comprising: (a) generating a three 
25 dimensional (3D) volumetric data set of a patient region 

within which the target structures reside from a plurality of 
segmented medical image slices; (b) grouping contiguous 
structures that are depicted in the 3D volumetric data set to 
create corresponding grouped structure data sets; (c) 
assigning each grouped structure data set to one of a 
plurality of detection algorithms, each detection algorithm 
being configured to detect a different type of target 
structure; and (d) processing each grouped structure data set 
according to its assigned detection algorithm to thereby 
detect whether any target structures are present in the 
medical images. 
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When used to detect pulmonary nodules, wherein the 
patient region being imaged is a region that includes a 
plurality of vessels, the method may further comprise 
classifying each grouped structure data set as either a vessel 
5 group data set or a non-vessel group data set, and wherein the 
assigning step comprises assigning each vessel group data set 
to a first detection algorithm and assigning each non-vessel 
group data set to a second detection algorithm. Further 
still, the processing step may comprise: (1) for each non- 
10 vessel group data set, determining a target structure status 
for the structure depicted therein based on geometric 
criteria; and (2) for each vessel group data set, (i) 
convolving the vessel group data set with at least one 
predetermined 3D morphological filter to thereby compute a 
15 correlation value between the vessel group data set and the 3D 
morphological filter, (ii) selecting a vessel group data set 
or subset thereof if its correlation value is within a 
predetermined range, and (iii) determining a target structure 
status for the structure depicted in the selected data set 
20 based on geometric criteria. 

While in the preferred embodiment the invention is 
practiced using CT images of a patient's lung region wherein 
the target structures are pulmonary nodules, it should be 
understood that the scope of the present invention extends in 
25 application to different imaging techniques such as magnetic 
resonance (MR) and ultrasound as well as different bodily 
areas. For example, the technique of the present invention 
may be used to detect bodily conditions other than pulmonary 
nodules such as colon polyps, aneurysms (both brain and 
30 otherwise), and a variety of other conditions. 

By classifying 3D structures found in the image data into 
different groups, the present invention allows for the 
particularized detection techniques to be applied to each 
group, wherein each detection technique is optimized to the 
35 specifics of its group. For example, because accurate 

detection of perivascular nodules in a patient's lung region 
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requires a more strenuous analysis than does detection of 
isolated nodules or pleural nodules, the present invention 
allows application of the more rigorous detection algorithm to 
the perivascular nodule candidates while providing application 
of the simpler detection algorithm to the other nodule 
candidates . Such a bifurcated approach preserves computing 
power and represents an efficient approach to CAD. 

Also, by application of 3D morphological filters to 
candidate structures that may include perivascular nodules, 
the present invention allows for a level of automated 
detection of perivascular nodules that, to the inventors' 
knowledge, has previously been unattainable. 

As would be appreciated by those having ordinary skill in 
the art upon review of the teachings herein, the present 
invention can be implemented in software, hardware, or some 
combination thereof. 

These and other features and advantages of the present 
invention are set forth below and in the enclosed figures. 

20 Brief Description of the Drawings : 

Figure 1 is a bar graph depicting the number and size 
distribution of lung nodules in 20 patients; 

Figure 2 is a flowchart depicting the process of the 
preferred embodiment of the present invention; 
25 Figures 3 (A) - (F) depict CT thorax images after various 

stages of thresholding and segmentation; 

Figures 4 (A) - (B) depict 3D volumetric data for subject 15 
before and after removal of the non- vessel group data; 

Figures 5 (A) - (D) depict CT images and the detection of 
30 nodules therein; 

Figures 6 (A) - (B) depict anterior and posterior views of 
the 3D volumetric representation if pulmonary vessels and 
detected lung nodules; 

Figure 7 is a bar graph depicting a comparison of the 
35 lung nodules detected by the present invention, by a chest 
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radiologist without the benefit of a CAD technique, and by a 
chest radiologist with the benefit of a CAD technique; and 

Figure 8 is a flowchart depicting the process of a colon 
polyp detection embodiment of the present invention. 

5 

Detailed Description of the Preferred Embodiment: 

The lung nodule detection algorithm of the preferred 
embodiment is a combination of 2D and 3D operations. While 
the 3D operation is more general and more powerful, in some 

10 image processing steps the 2D operation is preferred because 
it is more efficient and easier to implement. The combined 
2D/3D approach of the preferred embodiment represents an 
efficient technique for pulmonary nodule detection and may be 
used by radiologists when the CT images are reviewed for the 

15 presence of lung nodules. 

The CT images may be generated by a CT scanner as is well 
known in the art. The generated CT images are preferably 
electronically transferred from the scanner to a computer such 
as a PC workstation. The automated detection algorithm of the 

20 preferred embodiment is then executed on the CT image data by 
software . . 

In testing the preferred embodiment, a database of 20 
diagnostic thoracic CT scans from 2 0 patients (13 men, 7 
women; age range of 40-75 years with a mean age of 57 years) 

25 was used. These patients were consecutively identified by 
retrospectively reviewing a selecting clinical chest CT 
studies performed with a multislice CT scanner (a Plus 4 
Volume Zoom scanner manufactured by Siemens Medical of 
Erlangen, Germany) . The CT images were acquired with a spiral 

30 scanning technique using scan parameters of 120 KvP, 120 

effective mAs, a 0.5 s scan, 4x1 mm collimation, a standard 
high resolution lung kernel, 1 mm slice thickness, and a 1 mm 
reconstruction interval. Each image slice has a matrix size 
of 512x512 pixels with in-plane resolutions ranging from 0.55 

35 mm to 0.7 mm. 
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The 20 scans in the database together comprise a total of 
5,294 images, with the number of images per scan ranging from 
201 to 341 (with a mean of 265 images per scan) . A total of 
164 nodules were identified in the images by a chest 
5 radiologist who reviewed the CT images twice, with and without 
CAD results. The second reading of the CT images with CAD was 
conducted three weeks after the first reading (conducted 
without CAD) to ensure that no memory of the first reading 
would affect the second reading. Eighteen of the 2 0 patients 
10 had 1-13 pulmonary nodules (mean of 4.7), and the other two 

patients showed 25 and 54 nodules in their CT images. Several 
small nodules that were overlooked in the initial review were 
detected in the second review that was aided by CAD results. 
The results from these combined reviews were used as the gold 
15 standard to which the results of the preferred embodiment are 
compared. Figure 1 illustrates detailed number and size 
distributions of the pulmonary nodules for each subject. The 
number of nodules by size distribution was as follows: there 
were 27 nodules greater than 10 mm in diameter, there were 80 
20 nodules within the diameter range of 5-10 mm, and there were 
57 nodules within the diameter range of 3-5 mm. 

The PC workstation used in testing the preferred 
embodiment possessed 384 MB and Pentium III, 450 MHz 
processing. The software was implemented using MATLAB 
25 software, manufactured by The MathWorks, Inc. of Natick, 

Massachusetts. However, as should be understood by those of 
ordinary skill in the art, other processors may be used to 
carry out the processing tasks of the algorithm. Further 
still, it should be noted that these processing tasks can be 
30 performed in a distributed manner. As such, when the term 

"processor" is used herein, it encompasses not only a single 
processor that is configured to perform all tasks of the 
algorithm but also distributed processors, wherein each 
carries out at least a portion of the processing load for the 
35 algorithm. 
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Figure 2 illustrates the preferred algorithm of the 
present invention. When steps 100-106 of Figure 2 are 
performed in the 2D realm (slice by slice) and steps 108-116 
are performed in the 3D (volumetric) realm, it should be 
5 understood that the 2D steps could also be performed in the 3D 
volumetric realm. 

Similarly, it should be understood that the 3D steps can 
be performed in the 2D realm. In such a case, step 108 could 
be eliminated and steps 110-116 would be performed in a slice- 

10 by-slice mode. Nodule candidates would be evaluated for 2D 
geometric criteria, and candidates between adjacent slices 
would be compared and merged if those candidates satisfy a 
defined connectivity criteria. Further, classification would 
be preferably based on each nodule candidate's 2D features. 

15 Further still, it should be understood that individual ones of 
steps 110-116 can be performed in a 2D slice-by-slice mode 
rather than a 3D mode if desired by a practitioner of the 
invention. 

At step 100, CT images are generated by a scanner and 
20 received by a workstation executing software that implements 

the present invention. In the example described above, the CT 
image data that was initially acquired at 16 -bit grayscale 
resolution was reduced to 8 -bits to improve computing power. 
However, as would be understood by those of ordinary skill in 
25 the art, the present invention may also be implemented without 
such data reduction. 

The thorax's representation in the CT image slices can be 
easily delineated from background air because of its high 
tissue-to-air contrast. For this same reason, the lung's 
30 representation in the CT image slices can also be. distinctly 
separated from its surrounding soft tissues or bony tissues. 
Gray- level thresholding methods, which are known in the art, 
were used to create binary images and thereafter to detect the 
boundary of the thorax and lung regions (see the Armato 
article discussed above and also Giger et al . , Computerized 
Detection of Pulmonary Nodules in Computed Tomography Images, 
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Investigative Radiology 1994; 29: 459-65; Ko et al . , Chest CT: 
automated nodule detection and assessment of change over time 
- -preliminary experience, Radiology 2001; 218: 267-73; and Li 
et al., Computer-aided diagnostic scheme for lung nodule 
5 detection in digital chest radiographs by use of a multiple- 
template matching technique, Med Phys 2001; 28: 2070-76, the 
disclosures of all of which are incorporated herein by 
reference) . 

The threshold value may be determined from any of a 

10 variety of techniques. For example, a non- exhaustive list of 
techniques would include: (1) being determined from a fixed 
attenuation value, (2) being determined from an analysis of an 
image attenuation profile, or (3) being determined from an 
analysis of a 2D or 3D region histogram. With the preferred 

15 embodiment, segmentation of the thorax and lung region (step 
104) was based on a thresholding method applied to each image 
slice with the threshold being determined by analysis of the 
2D region histogram which showed distinct groups of pixels 
belonging to the thorax and background air (step 102) . The 

20 thoracic region was segmented from the thresholded binary 
image. Background pixels representing the air outside the 
thorax were clipped, and the morphological operations of 
erosion and dilation were applied to eliminate scattered 
background. A new gray- level histogram was acquired from the 

25 pixels in the entire thoracic region, and the threshold value 
for lung segmentation was determined to maximize the 
separation between two major peaks in the histogram. These 
boundary detection processes were repeated on the segmented 
thorax region to segment the lung region. 

30 The lung boundary detected by the gray- level thresholding 

technique often does not include pleural nodules because these 
nodules are contiguous with the body wall, and as such, are 
segmented as part of the body wall rather than the lung 
region. Further, central pulmonary vessels which may have 

35 perivascular nodules attached thereto may be excluded from the 
segmented lung region. To correct such segmentation 
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shortcomings, a morphological operation as described in the 
Armato et al . paper and in Sonka and Fitzpatrick, Handbook of 
Medical Imaging: Medical Image Processing and Analysis, Vol. 
2, SPIE, Bellingham, WA 2000 (the entire disclosure of which 
is incorporated herein by reference) is used to refine the 
lung boundary (step 106) . Circular "closing" filters of 
variable diameters are iteratively applied to the segmented 
lung region to capture the pleural nodules and to contain 
central pulmonary vessels. 

Figure 3 illustrates the results of the lung region 
segmentation process including lung boundary refinement. 
Figure 3 (A) shows a transaxial CT image of the thorax from 
Subject 5 (see Figure 1). The image of Figure 3(A) was gray- 
level thresholded to generate a binary image as shown in 
15 Figure 3(B). An initial uncorrected lung region was estimated 
as shown in Figure 3(C). After lung boundary refinement, the 
image shown in Figure 3(D) was created. The initial CT image 
that is enclosed by the lung boundary of Figure 3(D) is shown 
in Figure 3 (E) . Figure 3 (F) shows the soft-tissue structures 
within the lung region that were segmented by gray- level 
thresholding. The final segmented lung region contains the 
three types of lung nodules (isolated, pleural, and 
perivascular), blood vessels, pulmonary parenchyma, airway, 
and mediastinal structures. 

Once the boundary of the lung region has been refined, 
the 2D segmented lung regions are stacked to generate a 3D 
volumetric data set of the lung region (step 108) . At step 
110, region growing is applied to the 3D data set to group 
contiguous structures in three dimensions using a 
30 predetermined degree of connectivity. Preferably, 18- 

connectivity is applied to the voxels of the 3D data set, as 
testing has shown it to provide better image quality than 
lower connectivity (6 -connectivity) and no significant 
differences in quality than higher connectivity (26- 
35 connectivity) . The volume of each grouped structure was 
computed by counting the number of connected voxels and 
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converting that number into mm 3 . Figure 4(A) depicts a 
resulting volumetric data set created from a stack of 2D 
segmented lung images. This figure contains the above- 
described three types of nodules as well as blood vessels and 
5 noise voxels. 

Step 110 also encompasses classification or labeling of 
each grouped structure in the 3D data set as either part of a 
vessel group data set or a non-vessel group data set. Vessel 
group and non-vessel group data sets can be readily 
10 differentiated by criteria such as size and anatomical 

location. Vessel groups are much larger, more central, and 
more craniocaudally elongated than non-vessel groups. The 
vessel group data set corresponds to the pulmonary vessel tree 
(which is the largest connected structure in each lung) and 
15 includes any perivascular nodules that are attached thereto. 
The non-vessel group data set corresponds to remaining soft 
tissue structures, including both isolated nodules and pleural 
nodules. The non-vessel group data set is shown 
representationally by box 112 in Figure 2 while the vessel 
20 group data set is shown representationally by box 114. Figure 
4(B) depicts the vessel group structures (which includes 
perivascular nodules) which is a subset of the structures 
shown in Figure 4(A) - (the structures not connected to the 
pulmonary vessels having been removed in Figure 4 (B) ) . 
25 Detection of isolated nodules and pleural nodules within 

the non-vessel group data set occurs at step 116. The nodule 
status of the structures in the non-vessel group data set is 
determined on the basis of geometric criteria. Further , it is 
preferred that the geometric criteria be used to identify non- 
30 nodule structures rather than be used to identify nodule 

structures. The inventors have found that a "filtering out 
non-nodules" approach is more efficient in implementation than 
a "filtering in nodules" approach. The three preferred 
geometric criteria are size, compactness, and elongation. 

The upper size limit for a nodule can safely be defined 
as 3 cm in diameter (see Tuddenham, Glossary of terms for 
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thoracic radiology: recommendations of the Nomenclature 
Committee of the Fleischner Society, AJR 1984, 143: 509-17, 
the disclosure of which is incorporated herein by reference) . 
Thus, a structure whose volume is greater than 14.14 cm 3 (which 
5 equals (4/3) *7i* (l . 5) 3 ) can be deemed a non-nodule. The lower 
size limit for a nodule can be safely defined as 3 mm in 
diameter (or 14.14 mm 3 in volume). Using these volume limits 
as range thresholds, the structures in the vessel group data 
set can have their nodule status assessed on the basis of 

10 their calculated volumes. However, as should be understood to 
those of ordinary skill in the art, other upper and lower size 
limits may be used in the practice of the present invention 
depending upon the level of precision desired by the 
practitioner of the invention. 

!5 In addition to their general size, nodules are typically 

compact. As such, compactness is also preferably used as a 
geometric criteria for assessing nodule status. The 
compactness of each candidate structure is preferably computed 
as the ratio of its volume to the volume of the smallest 3D 

20 box that encloses the candidate structure (defined as the 

compactness value) . A candidate structure whose compactness 
value is less than 0 . 5 or greater than 1.5 is deemed a non- 
nodule . 

Lastly, nodules are typical spherical while lung vessels 
25 are typically elongated. Therefore, in addition to size and 
compactness, elongation may also be used to determine nodule 
status. In doing so, an elongation factor is computed for 
each candidate structure in both 2D and 3D. The 2D elongation 
factor is preferably computed as the distance ratio of the 
30 major axis to the minor axis of a rectangle/ellipse enclosing 
the candidate. The 3D elongation factor is preferably 
computed as the ratio of the maximum to the minimum 
eigenvalues from the coordinates of the candidate structure's 
voxels. For relatively round or spherical structures, the 
35 elongation factor is close to 1. In the preferred embodiment, 
a candidate structure having an elongation factor greater than 
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3 is determined to be a non-nodule. Preferably, candidate 
structures are compared first against the 2D elongation factor 
and any surviving candidates are compared against the 3D 
elongation factor, which preserves computation time (the 3D 
5 elongation processing requires more computational time, and by 
first passing candidates through the 2D elongation test, 
several candidates can be discarded and avoid the 3D 
elongation test entirely) . However, it should be noted that 
the present invention may be practiced wherein only the 3D 
10 elongation test is used. 

As a result of this geometric processing, any candidate 
structures that were not excluded by size, compactness, or 
elongation constraints are classified as nodules. 

Detection of perivascular nodules within the vessel group 
15 data set presents a more challenging problem. While 

perivascular nodules are distinguished morphologically from 
vessels on the basis of their generally spherical shape 
(vessels are more elongated) , detection can be troublesome 
because the perivascular nodules are attached to the pulmonary 
20 vessels. Unless they are disassociated from their attached 
vessels, the shape and geometric features of perivascular 
nodules cannot be independently characterized, which is to 
say, direct application of the geometric criteria of step 116 
is ineffective to identify perivascular nodules. 
25 The present invention (at step 118) uses a 3D mult i -level 

morphological matching method to identify and extract 
perivascular nodule candidates from the vessel group 
structures. In this process, 3D morphological filters are 
used. Each filter defines a volume of pixels in a specified 
30 shape. These filters preferably are spherical in shape. 

However, it should be noted that more complex shapes may be 
used for matching with candidate structures. Further, while a 
single filter may be used in the matching process, it is 
preferred that a candidate structure be matched against a 
35 plurality of different filters, with each filter being "tuned" 
to a different expected nodule shape. In the preferred 
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embodiment, four different sizes of spherical filters (3, 6, 
9, and 12 mm in diameter) are used to identify nodule 
candidates ranging from 3-3 0 mm. Candidate structures can be 
matched against the filters either in parallel or in a series 
5 as desired by the practitioner of the invention. While the 

tested embodiment used the filters in series, it is envisioned 
that improvements in efficiency may be obtained through 
parallel processing of filters. 

Although the 3 mm filter was the most sensitive of the 

10 four and was able to detect nodule candidates larger than 3 mm 
in diameter, the use of the other filters helped the 
identification of nodule candidates that were subject to high 
variability in pulmonary vessel thickness in different lung 
regions. Each filter was particularly sensitive in detecting 

15 nodule candidates whose diameters were close to that of the 
designated filter size. While increasing the number of 
filters improved the sensitivity of detecting nodule 
candidates, such an increase also increased computation time 
and the detection of false positive nodule candidates. Thus, 

20 as would be understood by those of ordinary skill in the art, 
a trade-off exists when selecting the number of morphological 
filters and more or fewer than four filters may be used 
depending upon the needs of the situation at hand. 

The morphological filters are convolved with the 3D data 

25 of the vessel group data set to compute a correlation value. 
High correlation indicates a close match in the shape of the 
target candidate and the shape of the filter. The correlation 
value between the 3D vessel group data set (I) and the 
morphological filter (F) can be expressed in terms of the Fast 

30 Fourier Transform (FFT) method: 

(' * ^XO - £ /(') F(x - /) = — § /(„, ) . exp(2^u ( .x//iJ F( W/ ) exp(2n/ W/ (/-jc)/ 

wherein u t is the i th voxel in the vessel group data set I, 
wherein the value u f is the f th voxel in the morphological 
35 filter F, wherein the values ni and n f are the number of voxels 
in each respective data set (vessel group and filter) . 
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After the correlation value is computed, the perivascular 
nodule candidates are separated from non- candidates by 
thresholding. The threshold value was empirically determined 
to be in the range 50%-80%, such that ranges of 50%-100% to 
5 80% to 100% would indicate a match between the filter shape 

and the target candidate. As with the number of morphological 
filters, lowering the correlation value threshold increases 
not only the sensitivity but also the computation time and the 
number of false positives. As such, a design trade-off exists 
10 whose resolution is within the skill of an ordinary artisan. 
After this matching process, region growing and labeling are 
once again applied to the remaining nodule candidates with 
each nodule candidates shape and geometric features (e.g. 
volume) being computed. Thereafter, the nodule candidates are 

15 processed at step 116 as described above wherein each 

candidate structure's nodule status is determined on the basis 
of the above-described geometric criteria. 

Figure 5 depicts examples of detected nodules using the 
present invention and the original transaxial CT images. 

20 Figure 5 (B) shows both a perivascular nodule (arrow) and an 

isolated nodule (arrow head) that are detected by the present 
invention within the CT image of Figure 5(A) for subject 5. 
Figure 5(D) shows two pleural nodules (arrows) detected within 
the CT image of Figure 5(C) . No false negative or false 

25 positive modules are detected. 

Figures 6 (A) and (B) depict a 3D volume rendering of 
multiple detected nodules together with a pulmonary vessel 
(6(A) being an anterior view and 6(B) being a posterior view) 
for subject 15. The detected nodules are identified by 

30 arrows. Subject 15 has a total of 13 nodules, 7 isolated, 3 
pleural, and 3 perivascular) . Two false positives were 
identified. 

Overall, the sensitivity for nodule detection in testing 
of the preferred embodiment, by nodule category, was 97.4% for 
35 isolated nodules (76/78), 92.3% for pleural nodules (48/52), 
and 94.1% for perivascular nodules (32/34). The sensitivity 
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of detection by nodule size was 92.6% for nodules greater than 
or equal to 10 nun (25/27), 98.8% for nodules 5-10 mm (79/80), 
and 91.2% for 3-5 mm nodules (52/57). 

Figure 7 plots a comparison of the number of nodules 
5 detected by the preferred embodiment and by a chest 

radiologist. The false positives per patient was 6.9 (or 
0.84/true nodule) for all nodules greater than or equal to 3 
mm and 4.0 (or 0.67/nodule) for nodules greater than or equal 
to 5 mm. Nearly all of the false positives were less than 5 
10 mm. As can be seen from Figure 7, lung nodule detection by 
the preferred embodiment was highly accurate for the 20 test 
subjects and better than the radiologist's first reading 
without CAD. The processing time in one software 
implementation was 20 minutes. However, the present inventors 
15 envision that this processing time can be reduced through 
improvements in software coding. 

The preferred slice thickness and reconstruction interval 
for the CT slices is 1 mm or less. However, it should be 
noted that the algorithm can accommodate different slice 
20 thicknesses and/or different reconstruction intervals. The 
slice thickness and reconstruction interval of a CT scan can 
be obtained from the header of the images. After the boundary 
of the lung region is refined as described above in connection 
with Figure 2, the 2D segmented lung regions can be stacked to 
25 generate a 3D volumetric data set of the lung region. If the 
reconstruction interval is larger than 1 mm, finer- resolution 
slices can be interpolated at every 1 mm using the slice 
neighboring above and the slice neighboring below and 
integrated into the expanded 3D volumetric dataset. 
30 The reconstruction interval information can also be used 

in the process of evaluating the nodule status of nodule 
candidates to adjust the parameters for geometric criteria 
such as elongation factor and compactness. If the 
reconstruction interval is greater than or equal to 3 mm, 
35 candidate structures were compared first against the 2D 

elongation factor and any surviving candidates can be compared 
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against the 3D elongation factor. In this process, the upper 
limit of the 3D elongation factor is preferably increased to 
5, while the 2D elongation factor remains at 3. The 
compactness in this scenario is also preferably increased to 
5 2, when the reconstruction interval is greater than or equal 
to 3 mm . 

As previously noted, the algorithm can be applied to 
target structures other than pulmonary nodules. For example, 
the algorithm can be used to detect colon polyps. Figure 8 is 

10 a flowchart depicting an embodiment of the algorithm directed 
toward detecting colon polyps. 

The colonic polyp detection algorithm of the present 
invention is a combination of 2D and 3D operations. While the 
3D operation is more general and more powerful, in some image 

15 processing steps the 2D operation is preferred because it is 
more efficient and easier to implement. The combined 2D/3D 
approach of the colon polyp detection algorithm represents an 
efficient technique for colonic polyp detection and may be 
used by radiologists when medical images such as CT images are 

20 reviewed for the presence of colonic polyps. Furthermore, CT 
images with a fine slice thickness and a fine reconstruction 
interval may result in voxels that are isotropic or near 
isotropic. In such a case, the 2D operation may be performed 
in any direction either on the original transverse slices or 

25 on reformatted sagittal or coronal slices. 

The colon is an air-filled structure, and as such, its 
representation in CT image slices 800 can be easily identified 
from its surrounding soft tissues or bony tissues. Gray-level 
thresholding methods which, which are known in the art as 

30 mentioned in connection with Figure 2, can be used to create 
binary images and thereafter to detect the boundary of the 
abdomen/pelvis and colon in the abdominal /pelvic CT (step 
802) . There are, however, other air-filled structures in the 
abdominal/pelvic CT such as the lung bases, stomach, and small 

35 intestine. 
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At step 804, all air-filled structures are identified by 
a gray-level thresholding method and segmented therefrom (, 
and thereafter the colon is preferably separated from other 
structures based on its anatomical location, size, and 
5 regional contiguity from the rectum. The rectum is the most 
caudal air- filled structure in the body and can be 
unequivocally identified. The colon, which is connected to 
the rectum, is subsequently segmented at step 804 by means of 
a topologic continuity constraint that is known in the art. 

10 A completely segmented colon should include, starting 

from the rectum, the sigmoid, left, transverse, and right 
colon. The anatomical location of the segmented colonic 
regions is preferably evaluated and used as a constraint to 
ensure the segmentation of the entire colon. if this 

15 constraint is not satisfied, the segmented result is 

preferably displayed and the operator is asked to revise it so 
that a completely segmented the boundary of the colon can be 
generated. 

The detected colon boundary corresponds to the boundary 

20 between the air within the colon and the soft- tissue 

structures surrounding the colonic lumen. The soft -tissue 
structures constitute the colon walls, haustral folds, and 
polyps (if there is a polyp) . With the detected boundary as a 
starting point, the colonic soft- tissue structures are 

25 preferably segmented at step 806 by means of a region-growing 
method such as a morphological operation. A preferred 
morphological operation comprises applying iteratively 
circular "closing" filters of variable diameters to the 
segmented colon boundary to capture the polyps and other soft- 

30 tissue structures. The region-growing method essentially 
expands the detected colonic boundary outward to include 
polyps and other soft-tissue structures surrounding the colon 
lumen. Instead of the 2D region-growing method, a 3D region- 
growing method may be applied to segment the colonic soft- 

35 tissue structures. While the 3D operation is more general and 
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more powerful, the 2D operation tends to be more efficient and 
easier to implement . 

The size of the morphological filtering affects the 
quality of the 3D dataset of the segmented colonic soft-tissue 
5 structures. With a small size for the filtering, the size of 
the 3D dataset will be small, but more specific in that it 
will include small structures immediately adjacent to the 
colonic lumen. With a larger size for the filtering, the size 
of the 3D dataset will be larger, including more folds and 

10 structures deeper into the colonic wall. The 3D datasets 
obtained by applying different sizes of the morphologic 
filters are preferably geometrically subtracted or regrouped 
to generate different candidate sets (step 808) . 

At step 810, each grouped structure in the data set is 

15 classified or labeled as either part of a first data 

classification or a second data classification. The first 
classification preferably encompasses grouped structures 
containing isolated polyp candidates, colonic folds, and 
colonic walls. The second classification preferably 

20 encompasses grouped structures containing polyp candidates 
that are attached to colonic folds or colonic walls. 

The structures labelled with the first classification are 
preferably grouped at step 812 prior to the application of 
polyp status evaluation at step 818. The structures labelled 

25 with the second classification are grouped at step 814 prior 

to undergoing a different polyp. status determination operation 
at step 816. 

At step 818, the polyps are preferably classified from 
the candidate sets based on the size, compactness, elongation 

30 factor, and a voxel intensity data for the candidate sets. 
The voxel intensity data is an indicator for the tissue 
composition of the image portion, as is known in the art. A 
variety of types of intensity data can be used in the practice 
of the algorithm, including statistical measures such as a 

35 mean intensity value, an attenuation distribution, a minimum 
intensity value, and/or a maximum intensity value. Any 
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candidate structures that are not classified as polyps (822) 
or non-polyps (820) at step 818 are preferably processed by 
means of a 3D multi- level morphological matching method (step 
816) . This method is similar to that used for the lung nodule 
5 classification. For the colon polyp classification, 3D multi- 
level morphological matching method enhances the 
identification and extraction of polyp candidates that are 
attached to the colonic folds or walls. 

After this matching process, region growing and labeling 

10 were once again applied to the remaining polyp candidates with 
each polyp candidates shape and geometric features being 
computed at step 818. 

While the present invention has been described above in 
relation to its preferred embodiment, various modifications 

15 may be made thereto that still fall within the invention's 
scope, as would be recognized by those of ordinary skill in 
the art. Such modifications to the invention will be 
recognizable upon review of the teachings herein. 
Accordingly, the full scope of the present invention is to be 

20 defined solely by the appended claims and their legal 
equivalents. 
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